On Emergent Models and Optimization of Parameters

There has been considerable interest on the “New Science” and its promises, but very few practical evidence has been presented to motivate the complex systems hype. However, it seems that the new approaches can give new insights. This paper shows how the new conceptual tools can help also in practical control engineering tasks as in optimization of parameters. Discussions here are closely related to another paper [4]. 1. COMPLEX SYSTEMS AND EMERGENCE Assume that a set of experts has been developing a sophisticated partial differential equation model for, say, some chemical reactor. Typically, such a model is based on partial differential equations – making this kind of model useful for simulation or control design purposes, it has to be approximated. The resulting lumped parameter model will typically have dozens of free parameters that cannot be exactly determined using physical knowledge, and some kind of parameter tuning has to be carried out. Validation of such a model against actual measurements, simplifying it, and detecting the actual relevance of the individual parameters can be an extremely difficult task, and tools that could help in this task would be invaluable. As presented in [4], theory of complex systems may give new tools when searching for new tools for mastering complicated automation systems. The key point is to look at the phenomena from a higher abstraction level, ignoring the details of the physical components. Rather than concentrating on the actual realizations of the dynamic signal, one looks at the statistical and static relationships between the “qualifiers”, or process parameters, and corresponding “qualities”, or process behaviors. It can be claimed that a higher-level statistical model emerges from the lower-level deterministic behaviors. Whereas the paper [4] concentrates on discussing the possibilities of this new view as an approach for developing novel description formalisms for mastering the complex automation systems on the global scale, this paper studies the benefits that can be gained on a very local scale, when looking at individual process or model components and trying to optimize their behaviors. When analyzing such an approach, one can note that the benefits are caused by the increased homogeneity of looking at the processes: No matter what is the physical structure of the underlying process, or what is the implementation of the model or how it is parameterized, the same way of looking at the problem can be applied. In this sense, general tools for analysis and manipulation of (sub)models can be implemented. As will be illustrated in what follows, this kind of tool would include an interface for running successive model simulations, monitoring of system responses, an evaluation block for extracting quality measures out of that data – and, finally, mechanisms for constructing a model for the data and for optimizing the parameters as proposed by the model. 2. FOCUS ON THE “LOW LEVEL” Finding out relationships between observations is a data mining problem where different kinds of pattern recognition methodologies are needed. One can assume that a Gaussian mixture model is appropriate for describing observations; however, when studying phenomena at the very lowest level, one can further simplify the starting point. One can assume that the data is unimodal, consisting of a single Gaussian cluster. In this case powerful tools become available: For example, the underlying substructures within the cluster can be assumed to be linear, and linear multivariate statistical methods are available [3]. Typically, simulation based stochastic optimization methods are more or less heuristic, often being formalizable in the framework of Markov Chain Monte Carlo (MCMC) simulation. In our case, because of the assumedly known functional structure, optimization now becomes simpler: First, the parameters are varied and the results are recorded, just as in MCMC methods in general, but after this, because of the continuity/differentiability assumption, the appropriate search direction can be calculated in an efficient way. The remaining problem is the noisy nature of measurements, but this problem can be attacked using the multivariate statistical methods. The objective is to model the data; how to construct models so that they would be well suited for optimization of the quality measures – these issues are discussed below. 2.1. ON MULTIVARIATE REGRESSION Assume that the data is unimodal, spanning a single multivariate Gaussian distribution. Further, assume that some of the variables are inputs and some are outputs, the n inputs being collected in the vector θ and the m outputs being collected in the vector q. In the special case that is studied in this paper, the inputs are the parameters, or the “qualifiers”, and the outputs are the “qualities” (see [4]):